Data Profiling Report

Dataset statistics

Number of variables	17
Number of observations	6730436
Missing cells	1275801
Missing cells (%)	1.1%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	4.4 GiB
Average record size in memory	705.0 B

Variable types

DateTime	2
Categorical	1
Text	7
Numeric	7

Alerts

`DestinationLat` is highly overall correlated with `OriginLat` and 1 other fields	High correlation
`DestinationLong` is highly overall correlated with `OriginLong` and 1 other fields	High correlation
`OriginLat` is highly overall correlated with `DestinationLat` and 1 other fields	High correlation
`OriginLong` is highly overall correlated with `DestinationLong` and 1 other fields	High correlation
`VehicleLocation.Latitude` is highly overall correlated with `DestinationLat` and 1 other fields	High correlation
`VehicleLocation.Longitude` is highly overall correlated with `DestinationLong` and 1 other fields	High correlation
`ExpectedArrivalTime` has 872302 (13.0%) missing values	Missing
`ScheduledArrivalTime` has 172333 (2.6%) missing values	Missing
`DistanceFromStop` has 376918 (5.6%) zeros	Zeros

Reproduction

Analysis started	2024-10-18 08:33:20.331476
Analysis finished	2024-10-18 08:36:54.932884
Duration	3 minutes and 34.6 seconds
Software version	ydata-profiling vv4.11.0
Download configuration	config.json

RecordedAtTime
Date

Distinct	218287
Distinct (%)	3.2%
Missing	0
Missing (%)	0.0%
Memory size	51.3 MiB

Minimum	2017-06-01 00:01:18
Maximum	2017-06-30 23:53:38

Histogram with fixed size bins (bins=50)

DirectionRef
Categorical

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	372.3 MiB

1	3384112
0	3346324

Length

Max length	1
Median length	1
Mean length	1
Min length	1

Characters and Unicode

Total characters	6730436
Distinct characters	2
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	0
2nd row	1
3rd row	0
4th row	0
5th row	1

Common Values

Value	Count	Frequency (%)
1	3384112	50.3%
0	3346324	49.7%

Length

Histogram of lengths of the category

Common Values (Plot)

Value	Count	Frequency (%)
1	3384112	50.3%
0	3346324	49.7%

Most occurring characters

Value	Count	Frequency (%)
1	3384112	50.3%
0	3346324	49.7%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	6730436	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
1	3384112	50.3%
0	3346324	49.7%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	6730436	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
1	3384112	50.3%
0	3346324	49.7%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	6730436	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
1	3384112	50.3%
0	3346324	49.7%

PublishedLineName
Text

Distinct	326
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	387.9 MiB

Length

Max length	8
Median length	3
Mean length	3.4291633
Min length	2

Characters and Unicode

Total characters	23079764
Distinct characters	20
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	B8
2nd row	S61
3rd row	Bx10
4th row	Q5
5th row	Bx1

Value	Count	Frequency (%)
b6	124896	1.9%
b41	101068	1.5%
q58	94390	1.4%
m15-sbs	86625	1.3%
q44-sbs	85033	1.3%
b35	84248	1.3%
q27	83823	1.2%
bx36	83653	1.2%
b82	80618	1.2%
m101	80422	1.2%
Other values (316)	5825660	86.6%

Most occurring characters

Value	Count	Frequency (%)
B	4009298	17.4%
1	2338086	10.1%
4	1781490	7.7%
S	1617710	7.0%
2	1437081	6.2%
x	1378807	6.0%
M	1363064	5.9%
6	1313638	5.7%
3	1306050	5.7%
5	1257513	5.4%
Other values (10)	5277027	22.9%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	23079764	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
B	4009298	17.4%
1	2338086	10.1%
4	1781490	7.7%
S	1617710	7.0%
2	1437081	6.2%
x	1378807	6.0%
M	1363064	5.9%
6	1313638	5.7%
3	1306050	5.7%
5	1257513	5.4%
Other values (10)	5277027	22.9%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	23079764	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
B	4009298	17.4%
1	2338086	10.1%
4	1781490	7.7%
S	1617710	7.0%
2	1437081	6.2%
x	1378807	6.0%
M	1363064	5.9%
6	1313638	5.7%
3	1306050	5.7%
5	1257513	5.4%
Other values (10)	5277027	22.9%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	23079764	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
B	4009298	17.4%
1	2338086	10.1%
4	1781490	7.7%
S	1617710	7.0%
2	1437081	6.2%
x	1378807	6.0%
M	1363064	5.9%
6	1313638	5.7%
3	1306050	5.7%
5	1257513	5.4%
Other values (10)	5277027	22.9%

OriginName
Text

Distinct	606
Distinct (%)	< 0.1%
Missing	63156
Missing (%)	0.9%
Memory size	492.4 MiB

Length

Max length	39
Median length	32
Mean length	20.139376
Min length	9

Characters and Unicode

Total characters	134274857
Distinct characters	61
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	4 AV/95 ST
2nd row	ST GEORGE FERRY/S61 & S91
3rd row	E 206 ST/BAINBRIDGE AV
4th row	TEARDROP/LAYOVER
5th row	RIVERDALE AV/W 231 ST

Value	Count	Frequency (%)
st	2503797	10.5%
av	2454334	10.3%
e	513193	2.2%
w	476915	2.0%
av/e	349285	1.5%
bl	305157	1.3%
	271726	1.1%
rd	233183	1.0%
av/w	227171	1.0%
george	194066	0.8%
Other values (907)	16215704	68.3%

Most occurring characters

Value	Count	Frequency (%)
	17131294	12.8%
A	12587959	9.4%
T	9160542	6.8%
S	8674611	6.5%
E	8367741	6.2%
R	7713606	5.7%
/	6588481	4.9%
L	5959760	4.4%
V	5624788	4.2%
N	5396222	4.0%
Other values (51)	47069853	35.1%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	134274857	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
	17131294	12.8%
A	12587959	9.4%
T	9160542	6.8%
S	8674611	6.5%
E	8367741	6.2%
R	7713606	5.7%
/	6588481	4.9%
L	5959760	4.4%
V	5624788	4.2%
N	5396222	4.0%
Other values (51)	47069853	35.1%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	134274857	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
	17131294	12.8%
A	12587959	9.4%
T	9160542	6.8%
S	8674611	6.5%
E	8367741	6.2%
R	7713606	5.7%
/	6588481	4.9%
L	5959760	4.4%
V	5624788	4.2%
N	5396222	4.0%
Other values (51)	47069853	35.1%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	134274857	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
	17131294	12.8%
A	12587959	9.4%
T	9160542	6.8%
S	8674611	6.5%
E	8367741	6.2%
R	7713606	5.7%
/	6588481	4.9%
L	5959760	4.4%
V	5624788	4.2%
N	5396222	4.0%
Other values (51)	47069853	35.1%

OriginLat
Real number (ℝ)

High correlation

Distinct	662
Distinct (%)	< 0.1%
Missing	63156
Missing (%)	0.9%
Infinite	0
Infinite (%)	0.0%
Mean	40.72961

Minimum	40.506882
Maximum	40.912365
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	51.3 MiB

Quantile statistics

Minimum	40.506882
5-th percentile	40.581245
Q1	40.660664
median	40.715233
Q3	40.807869
95-th percentile	40.868076
Maximum	40.912365
Range	0.405483
Interquartile range (IQR)	0.147205

Descriptive statistics

Standard deviation	0.090273681
Coefficient of variation (CV)	0.0022164141
Kurtosis	-0.84679197
Mean	40.72961
Median Absolute Deviation (MAD)	0.071579
Skewness	-0.013067871
Sum	2.7155571 × 10⁸
Variance	0.0081493375
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
40.704906	93956	1.4%
40.80323	92422	1.4%
40.761806	68468	1.0%
40.59351	68366	1.0%
40.731342	67586	1.0%
40.849113	64310	1.0%
40.609566	63932	0.9%
40.701748	63415	0.9%
40.729568	58892	0.9%
40.760429	54070	0.8%
Other values (652)	5971863	88.7%
(Missing)	63156	0.9%

Value	Count	Frequency (%)
40.506882	10666	0.2%
40.508942	5513	0.1%
40.510155	46	< 0.1%
40.523865	111	< 0.1%
40.526684	347	< 0.1%
40.526787	47	< 0.1%
40.526802	99	< 0.1%
40.526962	158	< 0.1%
40.526997	39	< 0.1%
40.527	2213	< 0.1%

Value	Count	Frequency (%)
40.912365	38692	0.6%
40.910107	27265	0.4%
40.903339	20772	0.3%
40.90266	20560	0.3%
40.900444	114	< 0.1%
40.893269	1211	< 0.1%
40.889969	11803	0.2%
40.888432	169	< 0.1%
40.888401	12191	0.2%
40.888393	37	< 0.1%

OriginLong
Real number (ℝ)

High correlation

Distinct	660
Distinct (%)	< 0.1%
Missing	63156
Missing (%)	0.9%
Infinite	0
Infinite (%)	0.0%
Mean	-73.93111

Minimum	-74.248062
Maximum	-73.701866
Zeros	0
Zeros (%)	0.0%
Negative	6667280
Negative (%)	99.1%
Memory size	51.3 MiB

Quantile statistics

Minimum	-74.248062
5-th percentile	-74.112297
Q1	-73.987373
median	-73.932449
Q3	-73.879936
95-th percentile	-73.781059
Maximum	-73.701866
Range	0.546196
Interquartile range (IQR)	0.107437

Descriptive statistics

Standard deviation	0.094275259
Coefficient of variation (CV)	-0.0012751771
Kurtosis	0.867808
Mean	-73.93111
Median Absolute Deviation (MAD)	0.054115
Skewness	-0.32947249
Sum	-4.9291941 × 10⁸
Variance	0.0088878245
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-73.793304	93956	1.4%
-73.932449	92422	1.4%
-73.878334	79718	1.2%
-73.829559	68468	1.0%
-73.993996	68366	1.0%
-73.990288	67586	1.0%
-73.937752	64310	1.0%
-73.921814	63932	0.9%
-73.802399	63415	0.9%
-73.990051	58892	0.9%
Other values (650)	5946215	88.3%
(Missing)	63156	0.9%

Value	Count	Frequency (%)
-74.248062	46	< 0.1%
-74.246948	5513	0.1%
-74.232979	10666	0.2%
-74.226295	30541	0.5%
-74.220856	2388	< 0.1%
-74.216454	145	< 0.1%
-74.208282	14	< 0.1%
-74.208115	10	< 0.1%
-74.207069	60	< 0.1%
-74.199394	1580	< 0.1%

Value	Count	Frequency (%)
-73.701866	23068	0.3%
-73.706421	17653	0.3%
-73.708687	7153	0.1%
-73.712791	110	< 0.1%
-73.720215	7624	0.1%
-73.723259	16634	0.2%
-73.7248	675	< 0.1%
-73.72506	3860	0.1%
-73.725151	738	< 0.1%
-73.726044	10910	0.2%

DestinationName
Text

Distinct	778
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	535.6 MiB

Length

Max length	68
Median length	55
Mean length	26.4411
Min length	4

Characters and Unicode

Total characters	177960128
Distinct characters	60
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	3 ?
Unique (%)	< 0.1%

Sample

1st row	BROWNSVILLE ROCKAWAY AV
2nd row	S I MALL YUKON AV
3rd row	RIVERDALE 263 ST
4th row	ROSEDALE LIRR STA via MERRICK
5th row	MOTT HAVEN 136 ST via CONCOURSE

Value	Count	Frequency (%)
via	4084130	11.4%
av	2689657	7.5%
st	2645218	7.4%
bus	475173	1.3%
sta	473147	1.3%
select	467312	1.3%
ltd	461782	1.3%
jamaica	401099	1.1%
bay	394768	1.1%
bl	335273	0.9%
Other values (761)	23448925	65.4%

Most occurring characters

Value	Count	Frequency (%)
	29363798	16.5%
A	12162410	6.8%
S	11606550	6.5%
T	10981081	6.2%
E	10201454	5.7%
R	9245861	5.2%
N	7784506	4.4%
L	7693210	4.3%
I	6705936	3.8%
O	6462285	3.6%
Other values (50)	65753037	36.9%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	177960128	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
	29363798	16.5%
A	12162410	6.8%
S	11606550	6.5%
T	10981081	6.2%
E	10201454	5.7%
R	9245861	5.2%
N	7784506	4.4%
L	7693210	4.3%
I	6705936	3.8%
O	6462285	3.6%
Other values (50)	65753037	36.9%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	177960128	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
	29363798	16.5%
A	12162410	6.8%
S	11606550	6.5%
T	10981081	6.2%
E	10201454	5.7%
R	9245861	5.2%
N	7784506	4.4%
L	7693210	4.3%
I	6705936	3.8%
O	6462285	3.6%
Other values (50)	65753037	36.9%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	177960128	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
	29363798	16.5%
A	12162410	6.8%
S	11606550	6.5%
T	10981081	6.2%
E	10201454	5.7%
R	9245861	5.2%
N	7784506	4.4%
L	7693210	4.3%
I	6705936	3.8%
O	6462285	3.6%
Other values (50)	65753037	36.9%

DestinationLat
Real number (ℝ)

High correlation

Distinct	530
Distinct (%)	< 0.1%
Missing	10346
Missing (%)	0.2%
Infinite	0
Infinite (%)	0.0%
Mean	40.728634

Minimum	40.508106
Maximum	40.912376
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	51.3 MiB

Quantile statistics

Minimum	40.508106
5-th percentile	40.580952
Q1	40.660854
median	40.713356
Q3	40.807545
95-th percentile	40.868797
Maximum	40.912376
Range	0.40427
Interquartile range (IQR)	0.146691

Descriptive statistics

Standard deviation	0.090072137
Coefficient of variation (CV)	0.0022115187
Kurtosis	-0.82145738
Mean	40.728634
Median Absolute Deviation (MAD)	0.069584
Skewness	0.0081833507
Sum	2.7370009 × 10⁸
Variance	0.0081129899
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
40.704933	94435	1.4%
40.809654	79753	1.2%
40.761745	70740	1.1%
40.643585	66830	1.0%
40.592949	63341	0.9%
40.609142	61441	0.9%
40.849033	59350	0.9%
40.699776	59280	0.9%
40.701683	57233	0.9%
40.80315	56679	0.8%
Other values (520)	6051008	89.9%

Value	Count	Frequency (%)
40.508106	9928	0.1%
40.508942	5630	0.1%
40.50901	308	< 0.1%
40.510319	10	< 0.1%
40.526684	190	< 0.1%
40.526745	25	< 0.1%
40.526806	2614	< 0.1%
40.526836	568	< 0.1%
40.527	99	< 0.1%
40.53006	32539	0.5%

Value	Count	Frequency (%)
40.912376	35818	0.5%
40.91008	31233	0.5%
40.903309	24316	0.4%
40.902779	20788	0.3%
40.900753	56	< 0.1%
40.893486	1259	< 0.1%
40.889843	11224	0.2%
40.888496	13720	0.2%
40.885086	17961	0.3%
40.881062	20774	0.3%

DestinationLong
Real number (ℝ)

High correlation

Distinct	531
Distinct (%)	< 0.1%
Missing	10346
Missing (%)	0.2%
Infinite	0
Infinite (%)	0.0%
Mean	-73.931565

Minimum	-74.248192
Maximum	-73.701385
Zeros	0
Zeros (%)	0.0%
Negative	6720090
Negative (%)	99.8%
Memory size	51.3 MiB

Quantile statistics

Minimum	-74.248192
5-th percentile	-74.113136
Q1	-73.989311
median	-73.932266
Q3	-73.878326
95-th percentile	-73.779564
Maximum	-73.701385
Range	0.546807
Interquartile range (IQR)	0.110985

Descriptive statistics

Standard deviation	0.095103662
Coefficient of variation (CV)	-0.0012863743
Kurtosis	0.81906665
Mean	-73.931565
Median Absolute Deviation (MAD)	0.055145
Skewness	-0.31051508
Sum	-4.9682677 × 10⁸
Variance	0.0090447066
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-73.79332	94435	1.4%
-73.92836	79753	1.2%
-73.829529	70740	1.1%
-74.072609	66830	1.0%
-73.993385	63341	0.9%
-73.92144	61441	0.9%
-73.937309	59350	0.9%
-73.941505	59280	0.9%
-73.802475	57233	0.9%
-73.932266	56679	0.8%
Other values (521)	6051008	89.9%

Value	Count	Frequency (%)
-74.248192	10	< 0.1%
-74.246948	5630	0.1%
-74.24675	308	< 0.1%
-74.230072	9928	0.1%
-74.226654	32539	0.5%
-74.220345	2504	< 0.1%
-74.208282	46	< 0.1%
-74.208115	74	< 0.1%
-74.206604	5	< 0.1%
-74.199394	8246	0.1%

Value	Count	Frequency (%)
-73.701385	2441	< 0.1%
-73.701454	21360	0.3%
-73.706589	19743	0.3%
-73.70871	7841	0.1%
-73.712563	184	< 0.1%
-73.720772	8479	0.1%
-73.723145	18350	0.3%
-73.725174	4330	0.1%
-73.725914	11636	0.2%
-73.727409	17074	0.3%

VehicleRef
Text

Distinct	5719
Distinct (%)	0.1%
Missing	0
Missing (%)	0.0%
Memory size	423.1 MiB

Length

Max length	10
Median length	9
Mean length	8.9097703
Min length	8

Characters and Unicode

Total characters	59966639
Distinct characters	18
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	25 ?
Unique (%)	< 0.1%

Sample

1st row	NYCT_430
2nd row	NYCT_8263
3rd row	NYCT_4223
4th row	NYCT_8422
5th row	NYCT_4710

Value	Count	Frequency (%)
nyct_5860	2978	< 0.1%
nyct_1238	2866	< 0.1%
nyct_6081	2857	< 0.1%
nyct_7107	2831	< 0.1%
nyct_1261	2827	< 0.1%
nyct_6061	2822	< 0.1%
nyct_6043	2808	< 0.1%
nyct_5997	2807	< 0.1%
nyct_5857	2806	< 0.1%
nyct_4573	2803	< 0.1%
Other values (5709)	6702031	99.6%

Most occurring characters

Value	Count	Frequency (%)
T	6730436	11.2%
C	6730436	11.2%
_	6730436	11.2%
N	6718018	11.2%
Y	6718018	11.2%
4	3437605	5.7%
7	3226239	5.4%
5	3123108	5.2%
6	2995867	5.0%
8	2778027	4.6%
Other values (8)	10778449	18.0%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	59966639	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
T	6730436	11.2%
C	6730436	11.2%
_	6730436	11.2%
N	6718018	11.2%
Y	6718018	11.2%
4	3437605	5.7%
7	3226239	5.4%
5	3123108	5.2%
6	2995867	5.0%
8	2778027	4.6%
Other values (8)	10778449	18.0%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	59966639	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
T	6730436	11.2%
C	6730436	11.2%
_	6730436	11.2%
N	6718018	11.2%
Y	6718018	11.2%
4	3437605	5.7%
7	3226239	5.4%
5	3123108	5.2%
6	2995867	5.0%
8	2778027	4.6%
Other values (8)	10778449	18.0%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	59966639	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
T	6730436	11.2%
C	6730436	11.2%
_	6730436	11.2%
N	6718018	11.2%
Y	6718018	11.2%
4	3437605	5.7%
7	3226239	5.4%
5	3123108	5.2%
6	2995867	5.0%
8	2778027	4.6%
Other values (8)	10778449	18.0%

VehicleLocation.Latitude
Real number (ℝ)

High correlation

Distinct	360308
Distinct (%)	5.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	40.728485

Minimum	40.502879
Maximum	40.932657
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	51.3 MiB

Quantile statistics

Minimum	40.502879
5-th percentile	40.592907
Q1	40.659481
median	40.723225
Q3	40.803181
95-th percentile	40.866339
Maximum	40.932657
Range	0.429778
Interquartile range (IQR)	0.1437

Descriptive statistics

Standard deviation	0.086818631
Coefficient of variation (CV)	0.002131644
Kurtosis	-0.92838433
Mean	40.728485
Median Absolute Deviation (MAD)	0.070183
Skewness	0.026693405
Sum	2.7412046 × 10⁸
Variance	0.0075374746
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
40.714687	3540	0.1%
40.643562	2829	< 0.1%
40.807584	2793	< 0.1%
40.73029	2535	< 0.1%
40.707646	2483	< 0.1%
40.820521	2482	< 0.1%
40.576997	2320	< 0.1%
40.593481	2317	< 0.1%
40.795177	2301	< 0.1%
40.730118	2274	< 0.1%
Other values (360298)	6704562	99.6%

Value	Count	Frequency (%)
40.502879	6	< 0.1%
40.502881	11	< 0.1%
40.502882	1	< 0.1%
40.502883	18	< 0.1%
40.502886	1	< 0.1%
40.50289	7	< 0.1%
40.502898	16	< 0.1%
40.502901	2	< 0.1%
40.502905	4	< 0.1%
40.50291	7	< 0.1%

Value	Count	Frequency (%)
40.932657	1	< 0.1%
40.932079	1	< 0.1%
40.932013	1	< 0.1%
40.932002	1	< 0.1%
40.927591	1	< 0.1%
40.918263	1	< 0.1%
40.917285	1	< 0.1%
40.916806	1	< 0.1%
40.916289	1	< 0.1%
40.912675	1	< 0.1%

VehicleLocation.Longitude
Real number (ℝ)

High correlation

Distinct	426793
Distinct (%)	6.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	-73.930712

Minimum	-74.252339
Maximum	-73.701414
Zeros	0
Zeros (%)	0.0%
Negative	6730436
Negative (%)	100.0%
Memory size	51.3 MiB

Quantile statistics

Minimum	-74.252339
5-th percentile	-74.100952
Q1	-73.979298
median	-73.936528
Q3	-73.882413
95-th percentile	-73.776967
Maximum	-73.701414
Range	0.550925
Interquartile range (IQR)	0.096885

Descriptive statistics

Standard deviation	0.089140224
Coefficient of variation (CV)	-0.0012057266
Kurtosis	0.77288539
Mean	-73.930712
Median Absolute Deviation (MAD)	0.045881
Skewness	-0.24948622
Sum	-4.9758593 × 10⁸
Variance	0.0079459796
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-73.831286	3543	0.1%
-73.990592	3460	0.1%
-74.072656	2821	< 0.1%
-73.990411	2781	< 0.1%
-73.795448	2483	< 0.1%
-73.851537	2469	< 0.1%
-73.994042	2343	< 0.1%
-73.973036	2286	< 0.1%
-73.991257	2238	< 0.1%
-73.939805	2236	< 0.1%
Other values (426783)	6703776	99.6%

Value	Count	Frequency (%)
-74.252339	1	< 0.1%
-74.252338	2	< 0.1%
-74.252337	3	< 0.1%
-74.252336	1	< 0.1%
-74.252335	4	< 0.1%
-74.252333	4	< 0.1%
-74.252328	3	< 0.1%
-74.252326	4	< 0.1%
-74.252325	3	< 0.1%
-74.252324	1	< 0.1%

Value	Count	Frequency (%)
-73.701414	6	< 0.1%
-73.701416	1	< 0.1%
-73.701417	2	< 0.1%
-73.701492	208	< 0.1%
-73.701494	6	< 0.1%
-73.701496	3	< 0.1%
-73.701499	1	< 0.1%
-73.701548	140	< 0.1%
-73.701551	29	< 0.1%
-73.701553	290	< 0.1%

NextStopPointName
Text

Distinct	10894
Distinct (%)	0.2%
Missing	7002
Missing (%)	0.1%
Memory size	491.1 MiB

Length

Max length	46
Median length	39
Mean length	19.558554
Min length	5

Characters and Unicode

Total characters	131500644
Distinct characters	72
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	361 ?
Unique (%)	< 0.1%

Sample

1st row	FOSTER AV/E 18 ST
2nd row	MERRYMOUNT ST/TRAVIS AV
3rd row	HENRY HUDSON PKY E/W 235 ST
4th row	HOOK CREEK BL/SUNRISE HY
5th row	GRAND CONCOURSE/E 196 ST

Value	Count	Frequency (%)
av	2701038	11.2%
st	2680521	11.1%
e	580864	2.4%
av/e	564519	2.3%
w	426581	1.8%
bl	341588	1.4%
av/w	312331	1.3%
rd	297410	1.2%
pl	199718	0.8%
3	151044	0.6%
Other values (5618)	15920884	65.9%

Most occurring characters

Value	Count	Frequency (%)
	17477062	13.3%
A	12264159	9.3%
T	8534919	6.5%
S	8445715	6.4%
E	7674584	5.8%
/	6695424	5.1%
R	6566497	5.0%
V	6409844	4.9%
N	5713648	4.3%
L	5292393	4.0%
Other values (62)	46426399	35.3%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	131500644	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
	17477062	13.3%
A	12264159	9.3%
T	8534919	6.5%
S	8445715	6.4%
E	7674584	5.8%
/	6695424	5.1%
R	6566497	5.0%
V	6409844	4.9%
N	5713648	4.3%
L	5292393	4.0%
Other values (62)	46426399	35.3%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	131500644	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
	17477062	13.3%
A	12264159	9.3%
T	8534919	6.5%
S	8445715	6.4%
E	7674584	5.8%
/	6695424	5.1%
R	6566497	5.0%
V	6409844	4.9%
N	5713648	4.3%
L	5292393	4.0%
Other values (62)	46426399	35.3%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	131500644	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
	17477062	13.3%
A	12264159	9.3%
T	8534919	6.5%
S	8445715	6.4%
E	7674584	5.8%
/	6695424	5.1%
R	6566497	5.0%
V	6409844	4.9%
N	5713648	4.3%
L	5292393	4.0%
Other values (62)	46426399	35.3%

ArrivalProximityText
Text

Distinct	210
Distinct (%)	< 0.1%
Missing	7002
Missing (%)	0.1%
Memory size	433.6 MiB

Length

Max length	15
Median length	14
Mean length	10.583211
Min length	7

Characters and Unicode

Total characters	71155523
Distinct characters	29
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	approaching
2nd row	approaching
3rd row	at stop
4th row	< 1 stop away
5th row	at stop

Value	Count	Frequency (%)
stop	3972243	26.0%
approaching	2535937	16.6%
away	2287603	15.0%
1	2072349	13.6%
	2072171	13.6%
at	1899894	12.4%
miles	215254	1.4%
0.6	38387	0.3%
0.5	26686	0.2%
0.7	23021	0.2%
Other values (203)	127160	0.8%

Most occurring characters

Value	Count	Frequency (%)
a	11546974	16.2%
p	9044117	12.7%
	8547271	12.0%
o	6508180	9.1%
t	5872137	8.3%
s	4187497	5.9%
i	2751191	3.9%
c	2535937	3.6%
r	2535937	3.6%
g	2535937	3.6%
Other values (19)	15090345	21.2%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	71155523	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
a	11546974	16.2%
p	9044117	12.7%
	8547271	12.0%
o	6508180	9.1%
t	5872137	8.3%
s	4187497	5.9%
i	2751191	3.9%
c	2535937	3.6%
r	2535937	3.6%
g	2535937	3.6%
Other values (19)	15090345	21.2%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	71155523	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
a	11546974	16.2%
p	9044117	12.7%
	8547271	12.0%
o	6508180	9.1%
t	5872137	8.3%
s	4187497	5.9%
i	2751191	3.9%
c	2535937	3.6%
r	2535937	3.6%
g	2535937	3.6%
Other values (19)	15090345	21.2%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	71155523	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
a	11546974	16.2%
p	9044117	12.7%
	8547271	12.0%
o	6508180	9.1%
t	5872137	8.3%
s	4187497	5.9%
i	2751191	3.9%
c	2535937	3.6%
r	2535937	3.6%
g	2535937	3.6%
Other values (19)	15090345	21.2%

DistanceFromStop
Real number (ℝ)

Zeros

Distinct	16378
Distinct (%)	0.2%
Missing	7002
Missing (%)	0.1%
Infinite	0
Infinite (%)	0.0%
Mean	225.88129

Minimum	0
Maximum	35910
Zeros	376918
Zeros (%)	5.6%
Negative	0
Negative (%)	0.0%
Memory size	51.3 MiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	22
median	89
Q3	198
95-th percentile	591
Maximum	35910
Range	35910
Interquartile range (IQR)	176

Descriptive statistics

Standard deviation	997.59162
Coefficient of variation (CV)	4.4164421
Kurtosis	466.18625
Mean	225.88129
Median Absolute Deviation (MAD)	76
Skewness	19.186191
Sum	1.5186979 × 10⁹
Variance	995189.03
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	376918	5.6%
3	102020	1.5%
4	86779	1.3%
5	81394	1.2%
2	79675	1.2%
7	76552	1.1%
6	75339	1.1%
8	73854	1.1%
9	70356	1.0%
10	69676	1.0%
Other values (16368)	5630871	83.7%

Value	Count	Frequency (%)
0	376918	5.6%
1	12357	0.2%
2	79675	1.2%
3	102020	1.5%
4	86779	1.3%
5	81394	1.2%
6	75339	1.1%
7	76552	1.1%
8	73854	1.1%
9	70356	1.0%

Value	Count	Frequency (%)
35910	1	< 0.1%
35868	1	< 0.1%
33608	3	< 0.1%
33607	3	< 0.1%
33605	5	< 0.1%
33604	5	< 0.1%
33601	2	< 0.1%
33596	8	< 0.1%
33593	2	< 0.1%
33588	1	< 0.1%

ExpectedArrivalTime
Date

Missing

Distinct	733218
Distinct (%)	12.5%
Missing	872302
Missing (%)	13.0%
Memory size	51.3 MiB

Minimum	2017-06-01 00:03:55
Maximum	2017-07-01 00:01:25

Histogram with fixed size bins (bins=50)

ScheduledArrivalTime
Text

Missing

Distinct	93232
Distinct (%)	1.4%
Missing	172333
Missing (%)	2.6%
Memory size	411.8 MiB

Length

Max length	8
Median length	8
Mean length	8
Min length	8

Characters and Unicode

Total characters	52464824
Distinct characters	11
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	2490 ?
Unique (%)	< 0.1%

Sample

1st row	24:06:14
2nd row	23:58:02
3rd row	24:00:53
4th row	24:03:00
5th row	23:59:38

Value	Count	Frequency (%)
08:47:00	2626	< 0.1%
08:29:00	2577	< 0.1%
08:17:00	2571	< 0.1%
08:07:00	2520	< 0.1%
08:05:00	2509	< 0.1%
08:14:00	2478	< 0.1%
07:55:00	2444	< 0.1%
07:59:00	2432	< 0.1%
07:25:00	2422	< 0.1%
08:25:00	2418	< 0.1%
Other values (93222)	6533106	99.6%

Most occurring characters

Value	Count	Frequency (%)
:	13116206	25.0%
0	8985015	17.1%
1	7365101	14.0%
2	4503566	8.6%
5	3655433	7.0%
4	3559613	6.8%
3	3555432	6.8%
7	2003873	3.8%
8	1998956	3.8%
6	1886992	3.6%

Most occurring categories

Value	Count	Frequency (%)
(unknown)	52464824	100.0%

Most frequent character per category

(unknown)

Value	Count	Frequency (%)
:	13116206	25.0%
0	8985015	17.1%
1	7365101	14.0%
2	4503566	8.6%
5	3655433	7.0%
4	3559613	6.8%
3	3555432	6.8%
7	2003873	3.8%
8	1998956	3.8%
6	1886992	3.6%

Most occurring scripts

Value	Count	Frequency (%)
(unknown)	52464824	100.0%

Most frequent character per script

(unknown)

Value	Count	Frequency (%)
:	13116206	25.0%
0	8985015	17.1%
1	7365101	14.0%
2	4503566	8.6%
5	3655433	7.0%
4	3559613	6.8%
3	3555432	6.8%
7	2003873	3.8%
8	1998956	3.8%
6	1886992	3.6%

Most occurring blocks

Value	Count	Frequency (%)
(unknown)	52464824	100.0%

Most frequent character per block

(unknown)

Value	Count	Frequency (%)
:	13116206	25.0%
0	8985015	17.1%
1	7365101	14.0%
2	4503566	8.6%
5	3655433	7.0%
4	3559613	6.8%
3	3555432	6.8%
7	2003873	3.8%
8	1998956	3.8%
6	1886992	3.6%

	DestinationLat	DestinationLong	DirectionRef	DistanceFromStop	OriginLat	OriginLong	VehicleLocation.Latitude	VehicleLocation.Longitude
DestinationLat	1.000	0.412	0.346	0.006	0.723	0.317	0.891	0.397
DestinationLong	0.412	1.000	0.402	-0.045	0.315	0.572	0.388	0.843
DirectionRef	0.346	0.402	1.000	0.015	0.347	0.384	0.018	0.036
DistanceFromStop	0.006	-0.045	0.015	1.000	0.014	-0.046	0.006	-0.049
OriginLat	0.723	0.315	0.347	0.014	1.000	0.400	0.888	0.391
OriginLong	0.317	0.572	0.384	-0.046	0.400	1.000	0.387	0.819
VehicleLocation.Latitude	0.891	0.388	0.018	0.006	0.888	0.387	1.000	0.442
VehicleLocation.Longitude	0.397	0.843	0.036	-0.049	0.391	0.819	0.442	1.000

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The correlation heatmap measures nullity correlation: how strongly the presence or absence of one variable affects the presence of another.

	RecordedAtTime	DirectionRef	PublishedLineName	OriginName	OriginLat	OriginLong	DestinationName	DestinationLat	DestinationLong	VehicleRef	VehicleLocation.Latitude	VehicleLocation.Longitude	NextStopPointName	ArrivalProximityText	DistanceFromStop	ExpectedArrivalTime	ScheduledArrivalTime
0	2017-06-01 00:03:34	0	B8	4 AV/95 ST	40.616104	-74.031143	BROWNSVILLE ROCKAWAY AV	40.656048	-73.907379	NYCT_430	40.635170	-73.960803	FOSTER AV/E 18 ST	approaching	76.0	2017-06-01 00:03:59	24:06:14
1	2017-06-01 00:03:43	1	S61	ST GEORGE FERRY/S61 & S91	40.643169	-74.073494	S I MALL YUKON AV	40.575935	-74.167686	NYCT_8263	40.590802	-74.158340	MERRYMOUNT ST/TRAVIS AV	approaching	62.0	2017-06-01 00:03:56	23:58:02
2	2017-06-01 00:03:49	0	Bx10	E 206 ST/BAINBRIDGE AV	40.875008	-73.880142	RIVERDALE 263 ST	40.912376	-73.902534	NYCT_4223	40.886010	-73.912647	HENRY HUDSON PKY E/W 235 ST	at stop	5.0	2017-06-01 00:03:56	24:00:53
3	2017-06-01 00:03:31	0	Q5	TEARDROP/LAYOVER	40.701748	-73.802399	ROSEDALE LIRR STA via MERRICK	40.666012	-73.735939	NYCT_8422	40.668002	-73.729348	HOOK CREEK BL/SUNRISE HY	< 1 stop away	267.0	2017-06-01 00:04:03	24:03:00
4	2017-06-01 00:03:22	1	Bx1	RIVERDALE AV/W 231 ST	40.881187	-73.909340	MOTT HAVEN 136 ST via CONCOURSE	40.809654	-73.928360	NYCT_4710	40.868134	-73.893032	GRAND CONCOURSE/E 196 ST	at stop	11.0	2017-06-01 00:03:56	23:59:38
5	2017-06-01 00:03:40	0	M1	4 AV/E 10 ST	40.731342	-73.990288	HARLEM 147 ST via MADISON	40.821110	-73.935898	NYCT_3831	40.792897	-73.950023	MADISON AV/E 106 ST	approaching	73.0	2017-06-01 00:03:56	24:02:35
6	2017-06-01 00:03:24	0	B31	GERRITSEN AV/GERRITSEN BEACH	40.587101	-73.918503	MIDWOOD KINGS HWY STA	40.608433	-73.957100	NYCT_4611	40.587024	-73.918623	GERRITSEN AV/GERRITSEN BEACH	at stop	0.0	NaN	24:08:00
7	2017-06-01 00:03:29	0	B83	GATEWAY CTR TERM/GATEWAY DR	40.652649	-73.877029	BWAY JCT VN SNDRN AV	40.678139	-73.903572	NYCT_4841	40.648801	-73.882682	PENNSYLVANIA AV/DELMAR LOOP N	< 1 stop away	196.0	2017-06-01 00:04:13	23:58:47
8	2017-06-01 00:03:27	0	B82	STILLWELL TERMINAL BUS LOOP	40.577080	-73.981293	SPRING CRK TWRS SEAVIEW AV via KINGS HWY	40.642990	-73.878326	NYCT_6592	40.632258	-73.918318	FLATLANDS AV/RALPH AV	approaching	35.0	2017-06-01 00:03:56	24:00:00
9	2017-06-01 00:03:51	1	S59	RICHMOND TER/PARK AV #3	40.640167	-74.130966	HYLAN BL	40.534260	-74.154213	NYCT_8279	40.590689	-74.165811	RICHMOND AV/NOME AV	approaching	31.0	2017-06-01 00:03:56	24:01:14

	RecordedAtTime	DirectionRef	PublishedLineName	OriginName	OriginLat	OriginLong	DestinationName	DestinationLat	DestinationLong	VehicleRef	VehicleLocation.Latitude	VehicleLocation.Longitude	NextStopPointName	ArrivalProximityText	DistanceFromStop	ExpectedArrivalTime	ScheduledArrivalTime
6730426	2017-06-30 23:53:08	0	M23-SBS	12 AV/W 23 ST	40.748718	-74.008110	SELECT BUS EAST SIDE AVENUE C CROSSTOWN	40.733006	-73.974594	NYCT_5859	40.749455	-74.008044	W 24 ST/12 AV	approaching	143.0	2017-06-30 23:53:39	23:30:16
6730427	2017-06-30 23:53:11	1	M15	E 126 ST/2 AV	40.803230	-73.932449	SOUTH FERRY via 2 AV	40.701611	-74.012230	NYCT_6107	40.712605	-73.992562	MADISON ST/MARKET ST	< 1 stop away	203.0	2017-06-30 23:54:57	23:46:22
6730428	2017-06-30 23:53:24	0	B65	SMITH ST/FULTON ST	40.691208	-73.987373	CROWN HTS RALPH AV	40.670197	-73.922546	NYCT_4931	40.675349	-73.927717	ROCHESTER AV/BERGEN ST	approaching	85.0	2017-06-30 23:53:49	23:30:18
6730429	2017-06-30 23:53:12	0	M104	W 41 ST/8 AV	40.756550	-73.990120	W HARLEM 129 ST via BWAY	40.814907	-73.955048	NYCT_6705	40.767813	-73.981383	Columbus Circle (does not stop)	approaching	44.0	2017-06-30 23:53:39	23:47:00
6730430	2017-06-30 23:53:20	0	Bx11	W 179 ST/BROADWAY	40.849113	-73.937752	W FARMS RD SOUTHERN BL	40.825272	-73.891426	NYCT_700	40.840354	-73.922210	W 170 ST/EDWARD L GRANT HY	approaching	123.0	2017-06-30 23:54:13	24:09:34
6730431	2017-06-30 23:53:37	0	B54	JAY ST/MYRTLE PLZ	40.694504	-73.987122	RIDGEWOOD TERM via MYRTLE	40.700527	-73.910149	NYCT_4442	40.699765	-73.911974	GATES AV/WYCKOFF AV	approaching	47.0	2017-06-30 23:53:52	23:44:12
6730432	2017-06-30 23:53:13	1	M79-SBS	E 79 ST/EAST END AV	40.770741	-73.948715	SELECT BUS W SIDE RIVERSIDE DR CROSSTOWN	40.784988	-73.982460	NYCT_5927	40.770681	-73.948759	E 79 ST/EAST END AV	at stop	0.0	NaN	24:02:00
6730433	2017-06-30 23:53:21	1	M5	BROADWAY/W 178 ST	40.848522	-73.937706	31 ST 6 AV	40.747791	-73.988831	NYCT_6388	40.820420	-73.955842	W 135 ST/RIVERSIDE DR	approaching	120.0	2017-06-30 23:54:27	23:44:16
6730434	2017-06-30 23:53:34	0	M4	W 32 ST/7 AV	40.749405	-73.991020	WASH HTS CABRINI BLV via MADSON via BWAY	40.859013	-73.934250	NYCT_6392	40.797009	-73.948954	CENTRAL PK N/5 AV	at stop	22.0	2017-06-30 23:53:42	23:50:00
6730435	2017-06-30 23:53:18	0	Bx2	LINCOLN AV/E 137 ST	40.809616	-73.928276	KNGSBRDG HTS FT INDEP ST via CONCOURSE	40.878746	-73.898033	NYCT_4750	40.817354	-73.922631	E 149 ST/MORRIS AV	approaching	37.0	2017-06-30 23:53:39	23:45:15

Overview

Variables

Common Values

Length

Common Values (Plot)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts

Most frequent character per script

(unknown)

Most occurring blocks

Most frequent character per block

(unknown)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts

Most frequent character per script

(unknown)

Most occurring blocks

Most frequent character per block

(unknown)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts

Most frequent character per script

(unknown)

Most occurring blocks

Most frequent character per block

(unknown)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts

Most frequent character per script

(unknown)

Most occurring blocks

Most frequent character per block

(unknown)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts

Most frequent character per script

(unknown)

Most occurring blocks

Most frequent character per block

(unknown)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts

Most frequent character per script

(unknown)

Most occurring blocks

Most frequent character per block

(unknown)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts

Most frequent character per script

(unknown)

Most occurring blocks

Most frequent character per block

(unknown)

Most occurring characters

Most occurring categories

Most frequent character per category

(unknown)

Most occurring scripts